Methods and Systems for Lesion Localization, Definition and Verification

ABSTRACT

A method and apparatus for lesion or organ definition for the purpose of radiation treatment planning localization and treatment position verification. The apparatus uses a combination of an ultrasound imaging system and a diagnostic imaging system to acquire localization ultrasound images referenced in the coordinate space of the diagnostic imaging system through the use of position sensing system. The method compares the location of the lesion in the localization ultrasound images with the position of the lesion in ultrasound images taken while the patient lies on the treatment table of a therapy treatment unit, suggests corrective measures to place the lesion in its intended treatment position and executes the correction upon confirmation from qualified personnel.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a method wherein the shape, form, position andconfiguration of a lesion or tumour, to be treated by a radiationtherapy device, may be ascertained with greater definition, in order tobetter design a treatment plan for its eradication. In accordance with afurther aspect, the invention also relates to method and apparatus forverification of the position of the lesion with respect to the radiationbeam or beams prior to the execution of a radiation treatment. Theinvention relates to a method wherein the size, location and dispositionof a tumour may be determined, updated and tracked prior to and duringtreatment therefor.

2. Description of Prior Art

The goal of modern day radiation therapy of cancerous tumours orlesions, is to eradicate the tumour while avoiding to the maximum extentpossible damage to healthy tissue and organs in the vicinity of thetumour. Since the large majority of tumours are radioresponsive, theycan be controlled or eradicated completely if a sufficient radiationdose is delivered to the tumour volume. However, the delivery of thenecessary tumourcidal dose may result in certain complications due todamage of healthy tissue that surround the tumour, or due to damage toother healthy body organs located in the proximity of the tumour.Conformal therapy is a radiation treatment approach which attempts tocombine accurate target localization with focused radiation delivery inorder to conform the high dose region closely to the region defined bythe outer surface of the tumour while minimizing the dose to surroundinghealthy tissue or adjacent healthy organs. Various conformal therapytechniques are well known in the art.

Conformal radiation therapy employs dedicated radiation units capable ofproducing highly energetic radiation beams of photons, electrons orother charged particles. The radiation unit typically has a radiationsource, which is typically mounted on a rotatable gantry of theradiation treatment unit. Through gantry rotation, the radiation sourceis rotated about the patient who is typically placed on a treatmenttable and the radiation beam is directed towards the tumour or lesion tobe treated. Various types of devices are used to conform the shape ofthe radiation treatment beam to encompass tightly the outline of thetumour as seen by the radiation treatment beam as it traverses thepatient's body into the tumour. An example of such a device is amultileaf collimator, which consists of a set of computer-controlledmovable leaves or fingers, which can be positioned individually in andout of the radiation beam in order to shape it to the tumour outline.Various types of radiation treatment planning systems can create aradiation treatment plan which, once implemented will deliver aspecified dose to the tumour while sparing the surrounding healthytissue or adjacent healthy organs.

The basic problem in conformal radiation therapy is knowing the locationof the target, or lesion or tumour, or alternatively of the healthyorgans with respect to the intended placement of the radiation beam orfield (I) prior to the design of a radiation treatment plan and (II) atthe time of the radiation treatment. Localization of the target volumewithin the patient prior to the design of a radiation treatment plan isperformed by acquiring a three-dimensional image of the patient with aconventional diagnostic imaging device such as computerized tomographic(“CT”) imaging device, a magnetic resonance imaging (“MRI”) device or apositron emission tomographic (“PET”) imaging device, as they are knownin the art. These sophisticated devices may be available from a varietyof manufacturers, such as GE Medical Systems, Marconi, Toshiba, Siemens,Phillips and others.

At the present time, when the treatment is initiated, both the patient'sposition and the position of the target within the patient at the timeof the radiation treatment are assumed to be grossly the same at as theywere at the time the treatment plan was created. However, if theposition of the target volume is not correctly determined (I) prior tothe treatment plan creation or (II) at the time of treatment, treatmentfailures can occur in a sense that the conformal dose of radiation maynot be delivered to the correct location within the patient's body.Failures of type (I) can occur if the conventional imaging modalityfails to reveal completely the shape, location and orientation of thetumour or lesion or organ of interest. This may occur since not allconventional diagnostic imaging devices adequately, completely or fullydetermine the exact shape, size and orientation of a tumour, resultingin that even with the use of the most up-to-date diagnostic imagingdevice, some tumours may not be fully diagnosed. Failures of type (II)can occur as a result of organ displacement (motion) from day to day,which may occur from a variety of factors, such as growth of the tumour,change in the patient physionomy due to weight loss, or even patientbreathing. Failures of type (II) can also occur from incorrectpositioning of the patient on the treatment table of the radiationtreatment unit.

To avoid the above failures, present day radiation treatment planstypically regard the target of the radiation to occupy a space in thepatient's body, which is larger than it really is, in order to ensurethat the smaller tumour or lesion, will fall within the larger volume.As a result, some healthy tissue or healthy organs surrounding thetumour or lesion will be irradiated with the maximum radiation doseintended for the tumour or target. Delivering the maximum radiation doseto a larger volume of healthy tissue or healthy organs may increase therisk of damaging these, and may for example, promote future cancers inthe healthy surrounding tissue. For this reason oncologists usingpresent conformal radiation therapy may decide to deliver a lowerradiation dose to the intended treatment volume in order to spare thenon-target tissue with the potential disadvantage of compromising thesuccess of the treatment by underdosing some portion of the targetorgan.

In an attempt to improve the localization of the lesion for thetreatment of prostate cancer and therefore rectify failures of type I, amethod was disclosed Holupka et al., U.S. Pat. No. 5,810,007 whichutilizes a transrectal probe to generate a two-dimensional ultrasoundimage. This image is then superimposed on an image acquired with aconventional diagnostic imaging device, such as CT scan. The imageregistration in the above said method requires the identification of atleast 2 fiducials visible in both the ultrasound image and the imageacquired with the conventional diagnostic imaging device. However, thefollowing shortcomings may limit the utility of the above said method:

1. The transrectal ultrasound probe may considerably displace the lesionor organ thus providing inaccurate information about the spatiallocation of the lesion at treatment time if at that time the transrectalprobe is not re-inserted. In any event, the insertion and removal of theprobe prior to initiating treatment may cause displacement of thelesion, adding further uncertainty to the localization of the tumour.Moreover, inserting the transrectal probe for each treatment session cancause significant patient discomfort, resulting in this method notgaining popularity with physicians.

2. Holupka provides only for two dimensional images, and assumes thatthe 2D ultrasound image and the image obtained with the conventionaldiagnostic imaging modality are acquired in the same plane. For thiscase two identifiable fiducials in both images would be sufficient toregister and superimpose the images. However, there is no certainty thatthe ultrasound image and the image from the conventional diagnosticimaging device are providing images in the same imaging planes andtherefore a deviation of one image from the plane of another mayconsiderably compromise the accuracy of the method.

3. The above said method registers and superimposes a two-dimensionalultrasound image onto a 2-dimensional image acquired with a conventionaldiagnostic imaging modality. Thus the ultrasound definition of thelesion is performed only in a single plane. For the purposes ofthree-dimensional conformal therapy, a two-dimensional definition of thelesion is incomplete and therefore inadequate since in other imagingplanes, the extent of the lesion volume may be larger or smaller.

4. Further, Holupka is of limited application since it may only be usedwith respect to a very limited number of tumours, such as of the rectum,lower large intestine, and of the prostate. It can not be used for othertype of tumours.

In attempt to rectify failures of type II, another system was proposedto verify the target or lesion position prior to a radiation treatmentsession by Carol, U.S. Pat. No. 5,411,026. The system comprises anultrasound imaging device to acquire at least one ultrasound image ofthe lesion in the patient's body and a device to indicate the positionof the ultrasound image generating device or probe with respect to theradiation therapy device. The above said system verifies that the actualposition of the lesion immediately before the treatment session conformsto the desired position of the lesion in the radiation treatment plan bycomparing the outlines of the outer surface of the lesion as defined onthe at least one ultrasound image to the outline of the outer surface ofthe lesion as defined on the at least one of the diagnostic imagesobtained by a computerized tomographic (“CT”) or alternatively bymagnetic resonance imaging (“MRI”) device and used for the design of theradiation treatment plan. However, the following shortcomings may limitthe utility of the above said system.

1. The appearance of the tumour or lesion or organ in the ultrasoundimage or images can have an appearance different from that of tumour orlesion or organ in the images obtained with conventional diagnosticdevices. Thus the process of comparing outlines of the outer surfaces ofthe tumour or lesion or organ as they appear in images obtained withdifferent imaging devices may be inaccurate since these surfaces can bedifferent both in appearance and extent. In other words, Carol comparesapples and oranges, which results in an incomplete assessment of thetumour. Since the trend in conformal treatment is towards more accuratespatial delivery of the exact dose of radiation, this shortcoming isquite significant.

2. Carol also does not address failures of type I whereby the diagnosticimages obtained with computed tomography or magnetic resonance imagingdevices do not reveal completely the location or the extent of thetumour or lesion or organ, due to the inherent limitation of saiddevices with respect to certain tumours in certain locations.Furthermore if the computed tomography or magnetic resonance diagnosticimages do not reveal, or completely reveal, the tumour or organ orlesion, Carol will lack the means to outline an outer surface to serveas a reference for the comparison to the outer surface of the tumour orlesion or organ outlined on the one or more ultrasound images.

In view of the above description of the prior art it is therefore anobject of the invention to provide an improved method and apparatus forradiation therapy treatments to decrease the rate of occurrence of theabove defined failures of type I and type II.

It is another object of the invention to provide a novel method andapparatus for accurate localization, sizing and definition of tumour orlesion or other organ volume in preparation for radiation therapy.

It is an object of the present invention to provide for the use ofultrasound imaging at the planning stage of a treatment plan;

It is a further object of the invention to provide an improved methodand apparatus for establishing an ultrasound image or plurality ofultrasound images for target definition and localization and correlatingthis image or plurality of ultrasound images to radiation therapysimulator images, obtained with conventional diagnostic imaging devicessuch as a computerized tomographic (“CT”) imaging device, a magneticresonance imaging (“MRI”) device or a positron emission imaging device(“PET”), or any other type, such as for example future types ofdiagnostic devices.

It is also an object of the present invention to provide a novel methodfor three-dimensional superposition of a three-dimensional ultrasoundimage of a lesion onto another three-dimensional lesion image, such asCT or MRI or another ultrasound image.

It is yet another object of the invention to provide an improved methodand apparatus for accurate positioning of the target relative toradiation therapy beams based on the registration of an ultrasound imageor plurality of ultrasound images acquired immediately before or afterthe acquisition of conventional diagnostic images to an ultrasound imageor plurality of images acquired immediately before a radiation treatmentsession.

The invention relates to a method and apparatus for (a) lesionlocalization and tumour or lesion or organ definition for radiotherapytreatment planning and (b) for verification and rectification of lesionposition during radiotherapy treatment.

SUMMARY OF THE INVENTION

In accordance with one aspect, the present invention may include anumber of steps to improve the localization, sizing, definition andorientation of a tumour or lesion or organ or of any other area of abody. Although the present invention may be contemplated for cancertreatment in humans, it is understood that it may also be used forother, non cancer treating medical applications, in both humans andanimals.

In accordance with a general aspect, the localization, sizing, etc. . .. of a tumour may be necessary in order to devise a treatment plan forthe treatment or eradication a tumour, or for any other necessary orrequired medical investigation. The steps may comprise: disposing thepatient on the table of the conventional diagnostic imaging device;acquiring a diagnostic image or plurality of diagnostic images, usingany known conventional diagnostic imaging device, such as for example, aCT, MRI or PET scan. Said acquisition may comprise the use of a numberof fiducials placed on the patient surface so that the geometricorientation of the diagnostic image or images can be determined withrespect to the diagnostic imaging device; acquiring an ultrasound imageor plurality of ultrasound images immediately before or immediatelyafter the acquisition of the diagnostic images with the ultrasound imagegenerating means being disposed in a known geometric orientation withrespect to the diagnostic imaging device for each ultrasound imagegenerated; superimposing (known in the art as fusing) or combining theultrasound image or images with the diagnostic image or images with theprevious knowledge about their geometric orientation: outlining thecontours of the outer surface of the tumour or lesion or organ on theultrasound image or images and simultaneously displaying the above saidouter surface on the diagnostic image or images; employing the abovesaid contours of the outer surface of the tumour or lesion or organ forthe design of a radiation treatment plan.

With respect to the verification of the tumour or lesion or organposition with respect to the radiation therapy device the presentinvention may include the steps of: disposing the patient on thetreatment table of a radiation therapy device; generating at least oneultrasound, i.e. US, image of the lesion in the patient's body with theUS image generating means, that is the probe, being disposed in a knowngeometric orientation for each US image generated; comparing the abovesaid ultrasound image or images to the ultrasound image or imagesobtained at the time of the acquisition of the diagnostic images wherebythe position of the tumour or lesion or organ with respect to theradiation therapy device may be verified to establish conformity withthe desired position of the tumour or lesion or organ in the radiationtreatment plan.

Another feature of the present invention may include the method ofcomparing or registering the ultrasound image or images acquiredimmediately before the radiation treatment session to the ultrasoundimage or images obtained immediately before or after the acquisition ofthe diagnostic images. This method may employ either gray-level imagecorrelation without the need of contour outlines or alternatively theregistration of geometric objects (as known in the art) composed of theoutlines the outer surface of the tumour or lesion or organ as definedon the ultrasound image or images acquired in the diagnostic and theradiation therapy room.

As a result of the above said image comparison another feature of thepresent invention is the step of determining the necessary tumour orlesion or organ displacement in order to dispose the tumour or lesion ororgan in the desired position prescribed by the radiation treatmentplan. A further feature of the present invention may include the step ofperforming the above determined tumour or lesion or organ displacementby but not restricted to, moving the treatment table with respect to theradiation treatment device, rotating the treatment table with respect tothe radiation treatment device, rotating the collimator of the radiationtreatment device as well as rotating the gantry of the radiation therapydevice, or any combination of the above.

Therefore, in accordance with one aspect of the present invention, thereis provided with:

a method for spatially localizing a tumour for the purposes of radiationtreatment planning comprising the steps of:

generating one or more diagnostic images of said tumour using adiagnostic imaging device selected from the group comprising a CAT scan,PET scan, CT scan,

assigning said tumour on said diagnostic image a first three-dimensionalcoordinate using an absolute coordinate reference system,

generating one or more ultrasound image of said tumour using anultrasound device

assigning said tumour on said ultrasound image a secondthree-dimensional coordinate using said absolute coordinate referencesystem,

fusing said ultrasound image and said image using said first and saidsecond three-dimensional coordinates so as to obtain an accurate imageof the tumour.

In accordance with a further embodiment, the present invention providesfor:

a method for spatially localizing a tumour for the purposes of radiationtreatment planning comprising the steps of:

placing on the patient a plurality of fiducials in proximity to theestimated position of said tumour,

assigning a first three-dimensional coordinate to said fiducials usingan absolute coordinate system,

generating one or more diagnostic images of said tumour using adiagnostic imaging device selected from the group comprising a CAT scan,PET scan, CT scan, said at least one diagnostic image comprising thereonan image of said tumour and further comprising said fiducials,

assigning said tumour on said at least one diagnostic image a secondthree-dimensional coordinate using said first three-dimensionalcoordinate of said fiducials as a reference,

generating one or more ultrasound image of said tumour using anultrasound device, said at least one ultrasound image comprising thereonan image of said tumour and further comprising said fiducials,

assigning said tumour on said ultrasound image a third three-dimensionalcoordinate using said first three-dimensional coordinate of saidfiducials as a reference,

fusing said ultrasound image and said image using said second and saidthird three-dimensional coordinates so as to obtain an accurate image ofthe tumour.

In accordance with yet a further aspect of the present invention, theremay be provided for

a method for spatially localizing a tumour for the purposes of radiationtreatment planning comprising the steps of:

placing on the patient a plurality of fiducials in proximity to theestimated position of said tumour,

assigning a first three-dimensional coordinate to said fiducials usingan absolute coordinate system,

generating one or more diagnostic images of said tumour using adiagnostic imaging device selected from the group comprising a CAT scan,PET scan, CT scan, said at least one diagnostic image comprising thereonan image of said tumour and further comprising said fiducials,

assigning said tumour on said at least one diagnostic image a secondthree-dimensional coordinate using said first three-dimensionalcoordinate of said fiducials as a reference,

generating one or more ultrasound image of said tumour using anultrasound device, said at least one ultrasound image comprising thereonan image of said tumour,

using a positioning system configured so as to allow the position andorientation of said one or more ultrasound image to be known, such thata tumour on said one or more ultrasound image may be assigned a thirdthree-dimensional coordinate in said absolute coordinate referencesystem,

fusing said ultrasound image and said image using said second and saidthird three-dimensional coordinates so as to obtain an accurate image ofthe tumour.

In accordance with another aspect of the present invention, there isprovided for:

a system for spatially localizing a tumour for the purposes of radiationtreatment planning comprising:

a diagnostic imaging device selected from the group comprising a CATscan, PET scan, CT scan, said diagnostic imaging device being adaptedfor generating at least one diagnostic image of said tumour,

an ultrasound device, said ultrasound device being adapted forgenerating at least one ultrasound image of said tumour,

a means for providing an absolute coordinate reference system, such thatsaid tumour is assigned with a first three-dimensional coordinate onsaid diagnostic image, and a second three-dimensional coordinates onsaid ultrasound image

a means for fusing said diagnostic image and said ultrasound image usingsaid first three-dimensional coordinate and said secondthree-dimensional coordinate so as to obtain an accurate image of saidtumour.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a functional block diagram of an embodiment of the presentinvention.

FIG. 2 is a perspective view of a conventional diagnostic imaging devicewith a patient schematically illustrated on the imaging table.

FIG. 3 is a further perspective view of an imaging device of FIG. 2.

FIG. 4 is an example of an image produced by the imaging device of FIG.2 illustrating the position of the lesion within the patient body.

FIG. 5 is an example of an image produced by the imaging device of FIG.2 illustrating fiducials with known positions with respect to thediagnostic imaging device and visible on the diagnostic image or images.

FIG. 6 is a representation of the three-dimensional diagnostic imagedata reconstructed from the multiple diagnostic images such as the onedepicted in FIG. S.

FIG. 7 is a perspective schematic view of the conventional diagnosticimaging device of FIG. 2, including a means for generating an ultrasoundimage of the lesion with the patient's body.

FIG. 8 is a representation of an ultrasound image of the tumour orlesion or organ.

FIG. 9 is a perspective view indicating multiple ultrasound images beingtaken of a lesion with the ultrasound apparatus of FIG. 7

FIG. 10 is a representation of the three-dimensional ultrasound imagedata reconstructed from the multiple ultrasound images acquired in theroom of the diagnostic imaging device and depicted in FIG. 9.

FIG. 11 is a representation of the three-dimensional ultrasound anddiagnostic image data sets superimposed or combined.

FIG. 12 is a representation of a sequence of two-dimensional ultrasoundpictures of the lesion within the three-dimensional ultrasound data withthe lesion having its outer surface outlined.

FIG. 13 is a three-dimensional rendering of the outline of the imageprepared from the lesion contours as illustrated in FIG. 12.

FIG. 14 is a perspective view of a conventional radiotherapy treatmentdevice, or linear accelerator including a rotatable couch, collimatorand gantry.

FIG. 15 is a perspective schematic view of the linear acceleratorincluding a means for generating an ultrasound image of the lesionwithin the patient's body.

FIG. 16 is a view indicating multiple ultrasound images being taken of alesion with the ultrasound imaging device of FIG. 15.

FIG. 17 is a representation of the three-dimensional ultrasound imagedata reconstructed from the multiple ultrasound images acquired in theroom of the therapy device and depicted in FIG. 16.

FIG. 18 is a representation of several two-dimensional ultrasound imageswith the lesion of FIG. 17 having its outer surface outlined.

FIG. 19 is a three-dimensional rendering of the outline of the imageprepared from the plurality of images from FIG. 18.

FIG. 20 is a representation of the process of determining the necessarycorrections in the treatment setup (table position, collimator andgantry rotation) prior to a treatment session based on contour orsurface registration.

FIG. 21 is a representation of the process of determining the necessarycorrections in the treatment setup (table position, collimator andgantry rotation) prior to a treatment session based on imagecross-correlation.

DETAILED DESCRIPTION OF THE INVENTION

An illustration of an embodiment of the method and apparatus of thepresent invention is shown in the components of the apparatus and imagesderived from the figures. In the schematic diagram of FIG. 1 theembodiment of the invention is generally illustrated. In order toachieve one of the objectives of the present invention, that is, toobtain the most accurate possible definition of the size, location andorientation of a tumour 010, it has been found that the target area of apatient's body 009 believed to comprise a tumour 010 may be scanned ordiagnosed using two distinct diagnostic apparatuses, and that theresulting images be compared. This may be achieved by comparing theimage of the tumour 010 acquired through the use of a diagnostic deviceselected from group comprising an MRI, CT or PET with the image of thetumour 010 obtained with an ultrasound apparatus, such as those ofAcuson, GE Medical Systems, Siemens, Toshiba and others. The order inwhich the two images is acquired is generally of no consequence, as longas the images are acquired within a short period of time of the other,for example, but not limited, to within one hour.

In accordance with this aspect, the first image to be acquired may forexample, be acquired through the use of a diagnostic imaging device 002,which may be, for example, a computerized tomography (“CT”) scanner, amagnetic resonance imaging (“MRI”) scanner or alternatively a positronemission tomography (“PET”) scanner, or any other equivalent device, orany other image producing diagnostic device. With reference to FIG. 2, a(conventional) diagnostic imaging device 002 is schematically shown witha conventional imaging table 001, upon which a patient 009 having atumour or a lesion or an organ of interest 010 may be disposed. Thediagnostic imaging device 002 may produce a cross-sectional image 023 ora “slice” of the body tissue, one such “slice” being schematicallyillustrated in FIG. 4, with the tumour or lesion or organ of interest010 shown. Several diagnostic images 023 may be acquired by causingrelative motion between the diagnostic imaging device 002 and thepatient 009 in the slice acquisition space 017 of the diagnostic imagingdevice 002 as shown, for example, in FIG. 3. FIG. 6 illustrates athree-dimensional picture 027 formed or reconstructed from a pluralityof (consecutive) diagnostic images 023 of parts or sections of thepatient 009.

Since the image of the tumour 010 as acquired with the diagnosticimaging device 002 is to be compared with the image of the same tumour010 taken with an ultrasound device 005, 008 as seen in FIG. 1, it isnecessary for the tumour to be referenced, i.e., given a set ofcoordinates which will allow said comparison to be effective. Forexample, said coordinates may be independent of both the diagnosticimaging device 002 and of the ultrasound device 005, 008. However, thecoordinate system may have to be able to correlate the position of atumour 010 found with the diagnostic imaging device 002 with theposition of the same tumour 010 found with the ultrasound imaging device005, 008. Therefore an absolute coordinate system 011 may need to beestablished.

To that end, a means for assigning an absolute coordinate to the tumour010 on an absolute coordinate system 011 may be provided, which absolutecoordinate may be used to correlate the, for example CT image of thetumour 010 with the ultrasound image thereof. In order to do so, anumber of markers, also known as fiducials, for example, three fiducials029 as illustrated in FIG. 5, may be placed on the patient's body 009 inor around the vicinity of the lesion before the acquisition of thediagnostic images 023. In accordance with one practice, the fiducials029 may all be placed in the same plane. The position of the fiducials(which may be any physical markers which may be easily seen oridentified in a diagnostic image) may then be ascertained in theabsolute coordinate system 011 through the use of any known system, forexample a measurement system.

The measurement system may take any known shape or form. For example,the measurement system may, in one embodiment, comprise one or morelasers, or laser systems, which lasers may for example be disposed onthe walls or the ceiling of the room in which the diagnostic imagingdevice 002 is located. Such measurement systems are known in the art,and may for example, be purchased commercially from a company calledCemar Electric, product Cermaligne, model number CL 505-CH2. The lasers,or any other suitable device, may be directed at the fiducials, andthrough the laser beams being bounced back off of the fiducials to theirsource or to any other measurement device, the coordinates of thefiducials may be determined and assigned. As a result, the fiducials maybe assigned absolute coordinates, for example, X₁, Y₁, Z₁, asillustrated in FIG. 5. The measurement system may then download orforward said absolute coordinates of the fiducials to the diagnosticimaging device.

From the slice 023 illustrated in FIG. 5, which shows both the fiducials029 and the tumour 010 on the same slice, it may then be possible toassign an absolute coordinates in the absolute coordinate system 011, toany point of the tumour 010. This may be done through a simplecorrelation based on the relative position of the fiducials 029 andtumour 010 as depicted in image slice 023 and measured in the imagecoordinate system 030 of slice 023. Since the coordinates of thefiducials 029 are known in the absolute coordinate system 011 and therelative position of the tumour 010 is known with respect to thefiducials 029 from information shown in slice 023 of FIG. 5, aconventional fitting algorithm known to those of ordinary skill in theart can be used to determine a transformation matrix, or coordinatetransformation so as to assign absolute coordinates to any point in thetumour 010, for example X₂, Y₂, Z₂. The determination of the coordinates(X, Y, Z) of any object within diagnostic image 023 or 027 may beaccomplished in this manner, and therefore assigned absolute coordinateswithin the absolute coordinate system 011.

Although FIGS. 4 and 5 arc shown as having the absolute coordinatesystem 011 disposed through the patient 009, it is understood that saidabsolute coordinate system 011 may be disposed otherwise than throughthe body 009.

An additional step in the determination of the size, location andorientation of a tumour 010 may be illustrated in FIG. 7, wherein ameans 005, 008 for generating at least one ultrasound image 016 of thelesion 010 is shown. Said means 008 for generating at least oneultrasound image may be disposed in the diagnostic imaging room wherethe diagnostic image device 002 is located. The means 008 for generatingan ultrasound image 016 may utilize a conventional, commerciallyavailable ultrasound probe 005. The ultrasound probe 005 may be broughtinto contact with the patient's body 009 in order to generate theultrasound image or images 016 of the tumour or lesion or organ 010, asillustrated in FIG. 8. As illustrated in FIG. 9, by moving, displacingor rotating the ultrasound probe 005, a plurality of ultrasound images016 of the tumour or lesion or organ 010 may be acquired in variousplanes. In FIG. 9, the lesion 010 is shown disposed within the pluralityof ultrasound images 016 with the plane of each ultrasound imagerepresentative of the orientation of the ultrasound probe 005 at thetime of the ultrasound image acquisition. As may be seen, the planes maynot necessarily be parallel to each other. From the plurality of theultrasound images 016 a reconstruction of the three-dimensional volumeor picture 031 (FIG. 10) of the ultrasound data may be performed.

In order to accurately reconstruct the three-dimensional volume 031 fromthe ultrasound data, and in order to assign an absolute coordinate inthe absolute coordinate system 011, the orientation and the position(hereafter referred to as the orientation) of the ultrasound probe 005with respect to the absolute coordinate system 011 must be known at thetime each ultrasound of the tumour 010 is made. In order to accomplishthis, a means 006 a, 006 b for indicating the (spatial) orientation ofthe ultrasound probe 005 may be used, and in particular may be disposedin the room of the diagnostic device 002 as shown in FIG. 7. Anyconventional position sensing system may be used as means 006 a, 006 bto determine the position and the orientation of the ultrasound probe005. For example, such systems are known in the art, sometimesgenerically called tracking systems, and may be available commerciallyfrom Ascension Technology Corporation, InterSense, Northern Digital Inc.Motion Analysis Corp. and others. The use of said position sensing means006 a and 006 b may enable the determination of the position of saidprobe with respect to the absolute coordinate system 011. For example,the positioning systems may include, but is not limited to: a camerasystem fixed in the room which looks at light emitting or reflectivemarkers mounted on the ultrasound probe 005; ultrasonic system withemitters mounted on the probe 005 with a detector measuring thedistances to these emitters by time measurements and consequentgeometric triangulation to determine the ultrasound probe 005 positionand orientation; a positioning system based on a mechanical arm with theultrasound probe 005 attached to the arm. It is to be noted that neitherthe ultrasound probe 005 nor the means 006 a, 006 b for indicating thegeometric orientation of the ultrasound probe 995 have to be fixed tothe table 001 of the diagnostic imaging device 002.

The means 006 a, 006 b for determining the coordinates and the geometricorientation of the ultrasound probe 005 are coordinated, aligned,connected or calibrated to the absolute coordinate system 011 i.e., forexample, the lasers. As a result of this alignment or calibration, thecoordinates (X, Y, Z) in the absolute coordinate reference system 011 ofany point or feature in an ultrasound images 016 may be ascertained. Inother words, the lasers which may form the basis of the absolutecoordinate system 011, may be used to determine the absolute coordinatesof a tumour 010 taken with an ultrasound image, as illustrated in FIG.10.

Because the absolute coordinate system 011 is common to both thediagnostic imaging device 002 and the ultrasound device 005, it ispossible to accurately correlate the position of a tumour 010 withrespect to both systems. With this knowledge, the value of theultrasound image data for each point within the reconstructed volume 031(FIG. 10) can be determined by interpolating algorithms known to thoseof ordinary skill in the art. The acquisition control and fusionsoftware may be executed on a dedicated computer or workstation 013 asillustrated in FIG. 1. Standard segmentation and other image enhancingtools are available to facilitate the process of lesion outlining andrendering.

Since the acquisition of the plurality of ultrasound images 016 is donebefore or immediately after (i.e. immediately before or immediatelyafter) the acquisition of the plurality of diagnostic images 023, theultrasound three-dimensional image data 031 and the diagnosticthree-dimensional image 027 represent pictures of spatially overlappingvolumes or sections of the patient anatomy at two very close moments oftime. For a large number of anatomical sites it can be assumed that,within the accuracy required for treatment planning, the patient anatomyat these two very close moments of time, does not change and thereforeboth the ultrasound three-dimensional image data 031 and the diagnosticthree-dimensional image data 027 represent temporally identical,spatially overlapping sections of the same patient anatomy. Given thatthe positions and the orientations of both the ultrasoundthree-dimensional image data 031 and the diagnostic three-dimensionalimage data 027 are each known with respect the absolute coordinatereference system 011 of the diagnostic device 022 the ultrasoundthree-dimensional image data 031 and the diagnostic three-dimensionalimage data 027 can be superimposed, i.e. accurately superimposed asillustrated in FIG. 11.

When the ultrasound three-dimensional image data 031 and the diagnosticthree-dimensional image data 027 are combined, contours 022 of the outersurface of the lesion 010 can be defined in arbitrarily selected planeswithin the ultrasound three-dimensional image data 031 or diagnosticthree-dimensional image data 027 (FIG. 12) and displayed at theircorrect location within the ultrasound three-dimensional image data 031or the diagnostic three-dimensional image data 027. These contours 022can be used to perform three-dimensional rendering 021 of the lesionwithin the diagnostic three-dimensional image data 027 (FIG. 13). Inthis manner, the lesion 010 is (1) localized and defined with respectthe absolute coordinate reference system 011 of the diagnostic device002 and (2) localized, defined and visualized within the diagnosticthree-dimensional image data 027. Because of (1) and (2) above, aradiation treatment plan can be designed in a conventional manner todeliver the necessary radiation to the lesion 010. This is so even ifthe lesion 010 may not have been completely visualized by the image orimages 023 acquired with the diagnostic imaging device 002 oralternately, by the ultrasound device 005. However, the combination ofthe two creates a more accurate picture of the tumour 010. Thereafter, aradiation treatment plan, such as for example a conformal plan, wherebythe shape of the radiation beam will conform to the spatial contour oroutline 022 of the lesion may be designed.

In addition, if a healthy organ 010 is localized and outlined with theabove described procedure, the radiation treatment plan will preferablybe designed to avoid excessive radiation damage to the organ 010. Theultrasound three-dimensional image data 031, the diagnosticthree-dimensional image data 027, the contours 022 of the outer surfaceof the lesion 010 and the three-dimensional rendering 021 of the lesion010 may then be transferred from the workstation 031 as illustrated inFIG. 1 to a computer or a workstation 014 in the control area of theradiation therapy device 003, also illustrated in FIG. 1, to serve asreference data for the verification of the treatment position of thetumour or lesion or organ 011 before the radiation treatment session.

It is understood that the above described comparison between adiagnostic image 027 and the ultrasound image 031 is not a necessarystep of the hereinafter described method. Thus, in accordance with anadditional embodiment of the present invention, and in order to avoidthe above described type II failures, it may be necessary to compare atumour 010 immediately prior to the beginning of the radiationtreatment, with the same tumour 010 as defined during the treatmentplan. This is to ensure that any change in the tumour, i.e. its size,location, orientation etc. . . . may be accounted for, through a changein the treatment plan if necessary. In order to accomplish this, anultrasound of the tumour 010 may be taken during the treatment plan, thewhole as described above, using ultrasound equipment 008 and 005. It isunderstood that the use of an absolute coordinate system 011 inconjunction with the taking of the ultrasound during the diagnosticphase may be required in order to assign absolute coordinates to saidtumour 010.

Before the radiation treatment session begins, the verification of thetumour or lesion or organ 010 position may proceed in the followingmanner. With reference to FIG. 14, the patient 009 having a tumour or alesion or an organ of interest 010 may be disposed on the treatmenttable 018 of the conventional therapy device 003 hereafter referred toas a linear accelerator. It is understood that the method hereindescribed may be used with any known or future radiation therapy device,or with any other type of therapy apparatus. The same patient has had inthe past, such as in the immediate past, an ultrasound performed inorder to determine the size, shape and orientation of the tumour 010during the diagnostic phase, the whole as described above. During saidultrasound, an absolute coordinate (X, Y, Z) was assigned to saidtumour. As depicted in FIG. 14, at the time of the treatment session, inthe therapy room, the position (possibly including orientation andshape), in other words, the absolute coordinates of the tumour or lesionor organ 010 of the patient on the therapy table 018 will undoubtedly bedifferent than the absolute coordinates of the tumour 010 as assignedduring the previous diagnostic phase. This may be due to a variety offactors, including different sizes and shapes of the machines involved,different positioning of the patient 009, and the fact that the tumour010 itself may have grown, shrunk, or moved.

It is therefore important to be able to account for, and compensate forthis difference in position of the tumour 010. In order to do so, acommon absolute reference frame or system, i.e. common to the ultrasounddevice 008 and to the therapy device 003 must be devised, to be able tocorrelate positions between a tumour 010 as identified by the ultrasoundimaging device 008, and the same tumour 010 identified by ultrasoundprior to being treated by the linear accelerator 003, which linearaccelerator is probably situated in a different physical location.

This may be accomplished through the use of a similar measurement systemas described above, which system may, for example, comprise lasersdisposed on the walls or the ceiling of the treatment room (019, FIG.1). The measurement system used in the diagnostic room with theultrasound 008 and 005 may be the same as the measurement system used inthe treatment room, although not strictly necessary. However, bothsystems must be calibrated so as to give a reference frame which iscommon to both the diagnostic ultrasound device 008 and the therapydevice 003. As a result, the absolute coordinate reference system 011 ofthe ultrasound diagnostic device 005 and the absolute coordinate system019 of the therapy device 003 (as illustrated in FIG. 17) may givecoordinates which are common to both, and which can be correlated. As aresult, the intended treatment position 032 (possibly includingorientation) of the lesion 010 may be calculated from the spatialcoordinates and extent of the lesion 010 determined previously by theultrasound imaging device 002 with the localization and definitionmethod described earlier and illustrated in FIG. 2 to FIG. 13.

Typically, in the process of treatment planning a 4×4 transformationmatrix T may be determined which when applied to the patient bymechanical motions of the therapy device table 018, of the treatmentdevice collimator 004 as well as of the treatment device gantry 007disposes the tumour or lesion or organ 010 in the desired treatmentposition. If the absolute coordinate reference system 011 of theultrasound diagnostic device 002 and the absolute coordinate system 019of the therapy device 003 are not identical, a predefined transformationmatrix or coordinate transformation may be used between the two tocorrelate coordinates of a tumour 010 in one system with the coordinatesin the other.

As a first step towards the verification of the intended treatmentposition, localization and definition of the actual position of thetumour, or lesion or organ 010 is performed in the room of theconventional radiotherapy device 003 similarly to the localization anddefinition of the tumour, or lesion or organ 010 performed in the roomof the ultrasound diagnostic device 002. A means 028 (FIG. 15) forgenerating at least one ultrasound image 020 of the lesion 010 (FIG.1-5) is disposed in the therapy room, as depicted in FIG. 15. Preferablythe means 028 for generating at least one ultrasound image 020 utilizesa conventional, commercially available ultrasound probe 025 (FIG. 15).

The ultrasound probe 025 is brought in contact with the patient body 009(FIG. 15) in order to generate an ultrasound image or images 020 of thetumour or lesion or organ 010 (FIG. 16). By moving or rotating theultrasound probe 025, a plurality of ultrasound images 020 (FIG. 16) ofthe tumour or lesion or organ 010 may be acquired. In FIG. 16, thelesion 010 is shown disposed within the plurality of ultrasound images020 with the plane of each ultrasound image representative of theorientation of the ultrasound probe 025 at the time of the ultrasoundimage acquisition. From the plurality of ultrasound images 020 areconstruction of the three-dimensional volume or picture 033 (FIG. 17)of the ultrasound data is performed in the absolute coordinate system019 of the therapy device 003. It is to be noted that, depending on thesize of the reconstructed volume 033 there may be location in theperiphery of reconstructed volume 033 for which ultrasound data are notavailable.

In order to accurately reconstruct the three-dimensional volume 033 ofthe ultrasound data from the plurality of ultrasound images 020, foreach acquired ultrasound image 020, the orientation and the position(hereafter referred to as the orientation) of the ultrasound probe 025with respect to the absolute coordinate system 019 of the therapy device003 must be known. A means 026 a, 026 b for indicating the geometricorientation of the ultrasound probe 025 may be disposed in the room ofthe therapy device 003 as shown in FIG. 15. Any conventional positionsensing system can be used as means 026 a, 026 b to determine theposition and the orientation of the ultrasound probe 025 with respect tothe coordinate system 019 of the therapy device 003, the whole as morefully described above. Although not necessarily identical to the systemdescribed above with respect to the diagnostic ultrasound device 008, itmay be convenient for both systems to be the same. It is to be notedthat neither the ultrasound probe 025 nor the means 026 a, 026 b forindicating the geometric orientation of the ultrasound probe 025 have tobe fixed to the table 018 of the therapy device 003.

The means 026 a-026 b for indicating the geometric orientation of theultrasound probe 025 are aligned with or as known in the art, calibratedto the absolute coordinate reference system 019 of the therapy device003. Because of this alignment or calibration, for any point or featurefrom the plurality of ultrasound images, the coordinates (A, B, C) ofany point, i.e. tumour 010 in the absolute coordinate system 019 of thetherapy device 003 are known. With this knowledge, the value of theultrasound image data for each point within the reconstructed volume 033(FIG. 17) can be determined by interpolating algorithms known to thoseof ordinary skill in the art. Furthermore, for any point or featurewithin the volume of ultrasound image data 033 (FIG. 17) the coordinates(X, Y, Z) in the absolute coordinate system 019 of the therapy device003 are known. Thus the localization of the tumour or lesion or organ001 as depicted by the three-dimensional ultrasound image data 033 (FIG.17) is complete. Furthermore, contours 024 (FIG. 18) of the outersurface of the lesion 010 can be defined in arbitrary planes within theultrasound three-dimensional image data 033 (FIG. 17). These contours024 can be used to properly perform three-dimensional rendering 034(FIG. 19) of the lesion in the coordinate system 019 of the therapydevice 003.

Once the tumour or lesion or organ 010 is localized in the room of thetherapy device 003, the necessary adjustments of the treatment table 018position, of the treatment device collimator 004 rotation as well as ofthe treatment device gantry 007 rotation can be performed by either ofthe following two methods. With reference to FIG. 20, the first methodIS establishes a coordinate transformation (4×4 transformation matrix) Rbetween the absolute coordinate system 011 of the ultrasound diagnosticdevice 002 and the coordinate system 019 of the therapy device 003 bysuperimposing or matching of the three-dimensional surface 022 orcontours 021 of the lesion 010 as outlined within the three-dimensionalultrasound localization data 031 acquired with the ultrasound diagnosticdevice 002 prior to the treatment plan to the three-dimensional surface034 or contours 024 of the lesion 010 as outlined within thethree-dimensional ultrasound localization data 033 acquired by theultrasound device 025 and 028 in the therapy device 003. Conventionalmethods for contour and surface matching may include chamfer matchingand “top-and-hat” least square distance matching, as well as any otherrequired or desired method.

An alternative method for the establishment of the coordinatetransformation R between the absolute coordinate system 031 of theultrasound diagnostic device 002 and the coordinate system 019 of thetherapy device 003, which does not rely on predefined contours orsurfaces is illustrated in FIG. 21. In this alternative, the image crosscorrelation is performed between the reconstructed three-dimensionalultrasound localization data 033 acquired in the room of the therapydevice 003 before the treatment session and the three-dimensionalultrasound localization data 031 acquired in the room of the diagnosticdevice 002 prior to the design of the treatment plan. The coordinatetransformation is selected to be the one which produces the highest peakof the correlation value between the two three-dimensional data sets 033and 031. The determination of the necessary adjustments of the treatmenttable 018 position, of the treatment device collimator 004 rotation aswell as of the treatment device gantry 007 rotation is then performed bya decomposition of the 4×4 transformation matrix TR-⁻¹ by algorithmsknown to those of ordinary skill in the art. It is to be noted thatafter the establishment of the coordinate transformation R between theabsolute coordinate system 011 of the ultrasound diagnostic device 002and the coordinate system 019 of the therapy device 003 by either of theabove said two methods, adjustments other than the above saidadjustments of the treatment table 018 position, of the treatment devicecollimator 004 rotation as well as of the treatment device gantry 007rotation can be undertaken. These may range from simple modifications ofthe initially intended radiation beam shapes to change in the beamintensities and even a completely new treatment plan with different beamarrangements. These adjustments are calculated with software running onthe workstation 014 and executed by the therapy device controller 015which is interfaced to the therapy device 003 and treatment tablecontroller 012 as illustrated in FIG. 1.

While particular preferred embodiments of the invention have been shownand described, it will be obvious to those of skill in the art thatchanges and modifications can be made without departing from the spiritand the scope of the invention as set forth in the claims. Accordingly,the invention is limited only by the scope of the appended claims.

1.-18. (canceled)
 19. A method of imaging a feature of a patient, themethod comprising: establishing a position of an ultrasound proberelative to a plane in the patient, the plane corresponding to anearlier-acquired diagnostic image of the patient; generating anultrasound image of the patient; identifying a feature on the ultrasoundimage; identifying the feature on the diagnostic image; registering theimages by transforming a coordinate system of one of the images to acoordinate system of the other image; and displaying the registeredimages.
 20. The method of claim 19 wherein registration comprisestransforming a coordinate system of the ultrasound image to a coordinatesystem of the diagnostic image.
 21. The method of claim 19 whereinregistration comprises transforming a coordinate system of thediagnostic image to a coordinate system of the ultrasound image.
 22. Themethod of claim 19 wherein the feature comprises one or more fiducials.23. The method of claim 19 wherein the feature comprises one or moreanatomical features of the patient.
 24. The method of claim 23 whereinthe anatomical feature comprises a lesion.
 25. The method of claim 19wherein displaying the images includes superimposing the diagnosticimage and the ultrasound image.
 26. The method of claim 19 wherein thediagnostic image comprises a non-ultrasound diagnostic image.
 27. Themethod of claim 19 wherein the transformation of the coordinate systemsof the one image and the other comprises using an absolute coordinatereference system.
 28. The method of claim 19 further comprisingdeveloping a radiation treatment plan based, at least in part, on theregistered images.
 29. The method of claim 19 further comprising guidingan instrument to the lesion using the registered images.
 30. The methodof claim 27 wherein the absolute reference coordinate system isestablished using a positional tracking system.
 31. The method of claim30 wherein said means for establishing an absolute coordinate referencesystem comprises at least one laser.
 32. The method of claim 31 whereinthe absolute coordinate reference system is independent of the deviceused to acquire the diagnostic image.
 33. The method of claim 32 whereinthe absolute coordinate reference system is independent of the deviceused to acquire the ultrasound image.
 34. The method of claim 19 whereinthe diagnostic image forms a three-dimensional image.
 35. The method ofclaim 19 further comprising the step of drawing one or more contours onone or more of the images.
 36. A system for simultaneously displaying anultrasound image and an earlier-acquired diagnostic image, the systemcomprising: a storage device for storing a diagnostic image acquiredfrom a MRI scanner, a PET scanner, a CT scanner or an ultrasoundscanner; an ultrasound device including an ultrasound probe forgenerating at least one ultrasound image of a portion of a patient; aposition tracker configured to track a position and an orientation ofthe ultrasound probe relative to the diagnostic image; means for fusingthe diagnostic image and the ultrasound image based on a feature commonto both images; and a display configured to display the registeredimages.
 37. The system of claim 36 wherein the means for registering thediagnostic image and the ultrasound image transforms a coordinate systemof the ultrasound image to a coordinate system of the diagnostic image.38. The system of claim 36 wherein the means for registering thediagnostic image and the ultrasound image transforms a coordinate systemof the diagnostic image to a coordinate system of the ultrasound image.39. The system of claim 36 wherein the ultrasound probe comprises one ormore markers responsive to the position tracker that facilitatedetermination of the positional and the orientation of the probe. 40.The system of claim 36 wherein the feature common to both imagescomprises one or more fiducials.
 41. The system of claim 36 wherein thefeature common to both images comprises one or more anatomical featuresof the patient.
 42. The system of claim 41 wherein the anatomicalfeature comprises a lesion.
 43. The system of claim 36 wherein thediagnostic image comprises a non-ultrasound diagnostic image.
 44. Thesystem of claim 36 wherein the means for registering the diagnosticimage and the ultrasound image transforms the coordinate systems of theimages using an absolute coordinate reference system.
 45. The system ofclaim 44 wherein the position tracker further establishes the absolutecoordinate reference system.
 46. The system of claim 44 wherein theposition tracker comprises at least one laser.
 47. A system forsimultaneously displaying an ultrasound image and an earlier-acquireddiagnostic image, the system comprising: a storage device for storing adiagnostic image acquired from a MRI scanner, a PET scanner, a CTscanner or an ultrasound scanner; an ultrasound device including anultrasound probe for generating at least one ultrasound image of aportion of a patient; a position tracker configured to track a positionand an orientation of the ultrasound probe relative to the diagnosticimage, wherein the ultrasound probe comprises one or more markersresponsive to the position tracker that facilitate determination of theposition and the orientation of the probe; a cross-correlator configuredto fuse the diagnostic image and the ultrasound image based on a featurecommon to both images, the feature comprising one or more fiducials orone or more anatomical features of the patient; and a display configuredto display the fused images.